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Abstract 



In this paper, we present a local convergence analysis of inexact Gauss-Newton like methods 
for solving nonlinear least squares problems. Under the hypothesis that the derivative of the 
function associated with the least square problem satisfies a majorant condition, we obtain that 
the method is well-defined and converges. Our analysis provides a clear relationship between 
the majorant function and the function associated with the least square problem. It also allows 
us to obtain an estimate of convergence ball for inexact Gauss-Newton like methods and some 
important, special cases. 

Keywords: Nonlinear least squares problems; inexact Gauss- Newton like methods; Majorant 
condition; Local convergence. 

1 Introduction 

Let X and Y be real or complex Hilbert spaces. Let Cl C X be an open set, and F : CI —> Y 
a continuously differentiable nonlinear function. Consider the following nonlinear least squares 
problems 



The interest in this problem arises in data fitting, when X = W 1 and Y = IR m and m is the number 
of observations and n is the number of parameters, see for example [E]. A solution x» G O of fl| 
is also called a least-squares solution of nonlinear equation F(x) = 0. 
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(1) 



When F'(x) is injective and has closed image for all x £ f2, the Gauss-Newton's method finds 
stationary points of the above problem. Formally, the Gauss-Newton's method is described as 
follows: Given an initial point xq £ ft, define 

x k+1 = x k + S k , F'(x k )*F'(x k )S k = -F'(x k )*F(x k ), A: = 0,1,..., 

where A* denotes the adjoint of the operator A. It is worth pointing out that if is solution of 
(PQ), F(x*) = and F'(x*) is invertible, then the theories of the Gauss-Newton's method merge 
into the theories of Newton's method. Early works dealing with the convergence of Newton's and 
Gauss-Newton's methods include [TJ El El El [H El [TUl HU HH HH [13 HH [20l EH EH ESI [28]. 

The inexact Gauss-Newton process is described as follows: Given an initial point S 0, define 

x k+1 =x k + S k , A; = 0,1,..., 

where B k : X — > Y is a linear operator and S k is any approximated solution of the linear system 

B k S k = -F'(x k )*F(x k ) + r k , 

for a suitable residual r k S Y. In particular, the above process is inexact Gauss-Newton method if 
B k = F'(x k ) T F'(x k ), the process is inexact modified Gauss-Newton method if B k = F' {xq) t F' {xq) , 
and it represents a inexact Gauss-Newton like method if B k is an approximation of F'(x k ) T F'(x k ). 

For inexact Newton methods, as shown in [12], if || r"^ || < #fc||-F(xfc)|| for k = 0, 1, . . . and {9 k } 
is a sequence of forcing terms such that < 9 k < 1 then there exists e > such that the sequence 
{x k }, for any initial point xo G -B(x*,e) = {x £ MJ 1 : \\x* — x|| < e}, is well defined and converges 
linearly to x* in the norm \\y\\* = \\F'(x*)y\\, where || || is any norm in W 1 . As pointed out by |22| 
(see also [23]) the result of [12] is difficult to apply due to a dependence of the norm || ||*, which is 
not computable. 

Formally, the inexact Gauss-Newton like methods for solving ([1]), which we will consider, are 
described as follows: Given an initial point xq £ fl, define 

x k+1 = x k + S k , B(x k )S k = -F'(x k )*F(x k ) + r k , A; = 0,1,..., 

where B{x k ) is a suitable invertible approximation of the derivative F'(x k )*F'(x k ) and the residual 
tolerance r k and the preconditioning invertible matrix P k (considered for the first time in |23| ) for 
the linear system defining the step S k satisfy 

\\Pkr k \\<e k \\P k F'{x k )*F(x k )\\, 

for suitable forcing number 9 k . Note that, if the forcing sequence vanishes, i.e., 9 k = for all k, 
the inexact Gauss-Newton methods include the class of Gauss-Newton iterative methods. Hence, 
the theories of inexact Gauss-Newton methods merge into the theories of Gauss-Newton methods. 

The classical local convergence analysis for the inexact Newton's methods (see |12|,l23j) requires, 
among other hypotheses, that F 1 satisfies the Lipschitz condition. In the last years, there have 
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been papers dealing with the issue of convergence of the Newton method and inexact Newton's 
method, including the Gauss-Newton's method and inexact Gauss-Newton's method, by relaxing 
the assumption of Lipschitz continuity of the derivative (see for example: [6j [H EH EH EH EH 
[2T| 128] ). One of the main conditions that relaxes the condition of the Lipschitz continuity of the 
derivative is the majorant condition, which we will use, and Wang's condition, introduced in |28j 
and used in [5j EJ El [H [201 [21] to study the Gauss-Newton's and Newton's methods. In fact, it can 
be shown that these conditions are equivalent. But the formulation as a majorant condition is in 
some sense better than Wang's condition, as it provides a clear relationship between the majorant 
function and the nonlinear function under consideration. Besides, the majorant condition provides 
a simpler proof of convergence. 

In the present paper, we are interested in the local convergence analysis, i.e., based on the 
information in a neighbourhood of a stationary point of ([T]) we determine the convergence ball of the 
method. Following the ideas of [14 11151 [TBI H7j. we will present a new local convergence analysis for 
inexact Gauss-Newton like methods under majorant condition. The convergence analysis presented 
provides a clear relationship between the majorant function, which relaxes the Lipschitz continuity 
of the derivate, and the function associated with the nonlinear least square problem (see for example: 
Lemmas [121 [T3l and fl4|) . Besides, the results presented here have the conditions and the proof of 
convergence in quite a simple manner. Moreover, two unrelated previous results pertaining to 
inexact Gauss-Newton like methods are unified, namely, the result for analytical functions and the 
classical one for functions with Lipschitz derivative. 

The organization of the paper is as follows. In Section 11.11 we list some notations and basic 
results used in our presentation. In Section [2] the main result is stated, and in Section 12.11 some 
properties involving the majorant function are established. In Section [2.21 we present the relation- 
ships between the majorant function and the non-linear function F. In Section [2.31 the main result 
is proven and some applications of this result are given in Section [3j Some final remarks are offered 
in Section [H 

1.1 Notation and auxiliary results 

The following notations and results are used throughout our presentation. Let X and Y be Hilbert 
spaces. The open and closed ball at a € X and radius 5 > are denoted, respectively by 

B(a, 5) := {x £ X; \\x — a\\ < 5}, B[a, 5] := {x £ X; \\x — a\\ ^ 5}. 

The set Q C X is an open set and the function F : Q — > Y is continuously differentiable, and F'{x) 
has closed image in fL 

Let A : X — > Y be a continuous and injective linear operator with closed image. The Moore- 
Penrose inverse ^ : Y -> X of A is defined by 

A^ := (A* A)' 1 A*, 

where A* denotes the adjoint of the linear operator A. 
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Lemma 1. (Banach's Lemma) Let B : X — > X be a continuous linear operator, and I : X — > X the 
identity operator. If \\B — I\\ < 1, then B is invertible and H-B -1 ]] < 1/ (1 — \\B — I\\) . 

Proof. See the proof of Lemma 1, p. 189 of Smale |26| with A = I and c = \\B — □ 

Lemma 2. Let A, B : X — > Y be a continuous linear operator with closed image. If A is injective, 
E = B — A and \\EA*\\ < 1, then B is injective. 

Proof. In fact, B = A + E = (I + EA^)A, from the condition ||.EL4J|| < 1, we have of Lemma [T] 
that / + EA^ is invertible. So, B is injective. □ 

The next lemma is proven in Stewart [27] ( see also, Wedin [29] ) for m x n matrix with m > n 
and rank(A) = rank(B) = n, that proof holds in more general context as we will state below. 

Lemma 3. Let A, B : X — > Y be continuous and injective linear operators with closed images. 
Assume that E = B - A and \\A^\\\\E\\ < 1, then 

Hutu < ''I'' „, listen <vwu^ 



1-IIAtHPII' 11 11 " l-||At||p||- 

Proposition 4. If < t < 1, then Y.Zoi' 1 + 2 )( { + = 2 /( 1 ~ tf- 

Proof. Take k = 2 in Lemma 3, pp. 161 of Blum, et al. [4]. □ 

Also, the following auxiliary results of elementary convex analysis will be needed: 
Proposition 5. Let R > 0. // tp : [0, R) — > R is convex, then 

D^(p(0) = hm u ^o+ = mf <„ • 

u u 

Proof. See Theorem 4.1.1 on pp. 21 of Hiriart-Urruty and Lemarechal |18j . □ 
Proposition 6. Let e > and r£ [0,1]. If (p : [0, e) — > R is convex, then I : (0, e) — > R defined by 

m = <P® - V (Tt) ^ 

is increasing. 

Proof. See Theorem 4.1.1 and Remark 4.1.2 on pp. 21 of Hiriart-Urruty and Lemarechal [18]. □ 
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2 Local analysis for inexact Gauss-Newton like methods 

In this section, we will state and prove a local theorem for inexact Gauss-Newton like methods. 
Assuming that the function 

Q3i4f(i)*F(i), 

has a point stationary x*, we will, under mild conditions, prove that the inexact Gauss-Newton like 
methods is well defined and that the generated sequence converges linearly to this point stationary. 
The statement of the theorem is as follows: 

Theorem 7. Let £1 C X be an open set, F : f2 — > Y a continuously differentiable function. Let 
E £1, R > and 

c := \\F(x*)\\, (3:= F'(x^ , n := sup {t € [0, R) : B(x„ t) C n} . 

Suppose that F'(x*)*F(x*) = 0, F'(x*) is infective and there exists a f : [0, R) — > R continuously 
differentiable such that 

\\F\x) - F'(x* + r(x - x*))|| < f (\\x - x,\\) - f (r\\x - x«.||) , (2) 

for all t £ [0, 1], x G B(x*, k) and 

hi) /(0) = and f (0) = -1; 

h2) /' is convex and strictly increasing; 

h3) a := V2c/3 2 D+/'(0) < 1. 

Take < i? < 1, < a;2 < oj\ such that uji(a + aft + i9) + W2 < 1. £e£ £/te positive constants 

v:=sup{t£ [0,i?):/3[/'(t) + l] < l} , 

p := sup <|t G (0, ,/) : (1 + "/i'^^f^ + 1] + u>i# + ^ < l} , r := min {«, p} . 

Then, the inexact Gauss-Newton like methods for solving (pQ), mi/i initial point xq G B{x*, r)\{x*} 

x k +i=x k + S k , B(x k )S k = -F'(x k )*F(x k ) +r k , A: = 0,1,..., (3) 

/or t/ie forcing term 6 k and the following conditions for the residual r k and the invertible matrix P k 
preconditioning the linear system in ([3]) 

\\P k r k \\ < e k \\P k F'(x k )*F(x k )\\, < e k cond(P k F f (x k )*F'(x k )) < 0, A; = 0, 1, ... , 
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where B(xk) is an invertible approximation of F' (x k )* F' (x k ) satisfying the following conditions 



B{x k )- 1 F J {x h )*F'{x k )\\<u) 1 , 



B(x k y l F'{x k yF'{x k )-I\\<u 2 



k = 0, 1,... 



is well defined, contained in B(x*,r), converges to x* and there holds 




the convergence of Gauss- Newton's like method under majorant condition which, for u\ = 1 and 
Ld2 = 0, i.e., B(x k ) = F'(x k )*F'(x k ), has been obtained by Ferreira et al. [16] in Theorem 7. Now, 
if taking c = (the so-called zero-residual case) and F'(x*) is invertible, we obtain the convergence 
of inexact Newton-Like methods under majorant condition, which has been obtained by Ferreira, 
Gongalves llty in Theorem 4- Finally, if c = $ = U2 = 0, u\ = 1 and F'{x*) is invertible in 
Theorem [7j we obtain the convergence of Newton's method under majorant condition, which has 
been obtained by Ferreira JJ4] in Theorem 2.1. 

For the important case 1? = 0, namely, Gauss-Newton's like method under majorant condition, 
the Theorem [7] becomes: 

Corollary 8. Let Q C X be an open set, F : Q — > Y a continuously differ entiable function. Let 
G fi, R > and 



Suppose that F'(x*)*F(x*) = 0, F'(x#) is injective and there exists a f : [0, R) — > M. continuously 
differentiable such that 



c := 



F(x*)\\ 



/3:= F'(xJ 



K := sup {t G [0, R) : B{x*,t) C 0} . 



F'{x) — F'{x* + t(x — x 



*))|| <f'{\\x-x4)-f{r\\x-x4), 



for all t G [0, 1], x G B(x*, k) and 



hi) /(0) = and /'(0) = -1; 



h2) /' is convex and strictly increasing; 
h3) a := V2c/3 2 L>+/'(0) < 1. 
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Take < co 2 < ^i swc/t i/iai wia + cj 2 < 1. Let v := sup {i G [0, i?) : /3[f'(t) + 1] < 1} , 

p := sup | t G (0,i/) : wi/3 1 _ ^7777^ + jvi + W2 < V' r := mm K Pi • 

Then, the Gauss-Newton's like method for solving (pQ) ; with initial point xq G -B(x*, 
= x fc + 5 fc , B(x k )S k = -F'(x k )*F(x k ), k = 0,1,..., 
where B(x k ) is an invertible approximation of F'(x k )*F'(x k ) satisfying 

\\B(x k )- 1 F'(x k )*F , (x k )\\ < CO!, \\B(x k )- 1 F , (x k )*F'(x k ) - I\\ < w 2 , fe = 0, 1, . . . , 
is we// defined, contained in B(x*,r), converges to x* and i/iere /io/ds 

n ^ , , ^/'(Iko ~ x*\\)\\xo ~ x *\\ ~ /(Iko ~ x *\\)} m„ , M 2 
" ^"-^ ||xo-^P[l-/3(/'(||xo-x,||) + l)] l]Xk X * 11 

+ ( v l|xo-x,||[l- /3 (/'(||xo-x,||) + l)] +W2 J ll^-^H' & = '^--- ( 5 ) 

Remark 2. Despite the fact that the above corollary is a special case of Theorem [?| the results 
contained therein extend the results of Chen and Li in as the results obtained /#/ are only for 
the case c = 0. 

Remark 3. Assumption ([2]) is crucial for our analysis. It should be pointed that, under appro- 
priate regularity conditions in the nonlinear function F, assumption ([2]) always holds on a suitable 
neighbourhood of x*. For instance, if F is two times continuously differentiable, then the majorant 
function f : [0, k) — > R, as defined by f(t) = Kt 2 /2 - t, where K = sup{||F"(x)|| : x G B{x*,k}} 
satisfies assumption ([2]). Estimating the constant K is a very difficult problem. Therefore, the goal 
is to identify classes of nonlinear functions for which it is possible to obtain a majorant function. 
We will give some examples of such classes in Section [3J 

To prove Theorem [7] we need some results. From here on, we assume that all assumptions of 
Theorem [7] hold. 



2.1 The majorant function 

In this section, we will prove that the constant k associated with Q and the constants v, p and r 
associated with the majorant function / are positive. We will also prove some results related to 
the function /. 

We begin by noting that k > 0, because Q is an open set and x* G f2. 
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Proposition 9. The constant v is positive and there holds 

P[f'(t) + 1] < 1, te(0,u). 

Proof. As f is continuous in (0, R) and /'(0) = —1, it is easy to conclude that 

lim/?[/'(i) + l] = 0. 

Thus, there exists a 6 > such that j3{f'{t) + 1) < 1 for all t £ (0,5). Hence, ^ > 0. 
Using h2 and definition of the last part of the proposition follows. 

Proposition 10. The following functions are increasing: 
i) [0 ,R) 3t ^l/[l-/3(f(t) + l)}; 
ii) (0, R)3t^[tf(t)-f(t)]/t 2 - 
iii) (0, R) 3 t ^ [f'(t) + l]/t; 

As a consequence, there is an increase of the following functions 



! 2 [l-/3(/'(t) + l)]' " ' t[l - W(t) + 1)]' 

Proof. The item i is immediate, because /' is strictly increasing in [0,R). 

For proving item ii, note that after some simple algebraic manipulations we have 

tfit) - f(t) _ f 1 f'(t) - f'(rt) ^_ 



□ 



t 2 h t 

So, applying Proposition [6] with f' = (p and e = R the statement follows. 

For establishing item iii use h2, /'(0) = —1 and Proposition [6] with f' = ip,e = R and r = 0. 

To prove that the functions in the last part are increasing, combine item i with ii for the first 
function, and i with iii for the second function. □ 

Proposition 11. The constant p is positive and there holds 
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Proof. First of all, note that the assumption hi implies, after simple calculation, that 

tf(t) - f(t) 



lim 



lim m - m >- /(Q))A _ Q 



t^o t[i - pint) + 1)] t-> i - (3(f(t) + 1) 

Again, using hi, some algebraic manipulation and that /' is convex, we have by Proposition [5] 



lim 



/'(«) + 1 



t^o t[l - p(f'{t) + 1)] t-*o 1 - (3(f'(t) + 1) 
Hence, by combining the two above equalities it is easy to conclude that 

,</'(() - /(<) + V2cj3[f'(t) + 1] 



linjd^M,- ([1 _ w , (t) + 1)] 



+ wii? + co 2 = (1 + T})uj 1 V2cf3 2 D + f'(0) + urf + 



UJ 2 . 



As, a = V2c/3 2 D + f(0) and w x (a + cn? + tf) +u 2 < 1, we obtain that there exists a (5 > such that 

Hence, 5 < p, which proves the first statement. To conclude the proof, we use the definition of p, 
the above inequality, and the last part of Proposition [TO]. □ 

2.2 Relationship of the majorant function with the non-linear function 

In this section we will present the main relationships between the majorant function / and the 
function F associated with the nonlinear least square problem. 

Lemma 12. Let x Eft. If \\x — x*\\ < min{z^, k}, then F'(x)*F'(x) is invertible and the following 
inequalities hold 



F'(x) t 



< 



i-p[f(\\x-x4) + iy 

In particular, F'(x)*F'(x) is invertible in B{x*,r). 



F'(xY - F\x*) 



< 



V2/? 2 [/'(||x-x,||) + l] 

i-/3[/'(i|x-x*n)+ir 



Proof. Let x £ Cl such that ||x — x*\\ < min{^, k}. Since \\x — < v, using the definition of /3, 
the inequality ([2D and last part of Proposition Owe have 

\\F'(x) - F'(x*)\\\\F>(xJ\\ < (3[f>(\\ x - x4) - /'(())] < 1. 

For the sake of simplicity, the notations define the following matrices 

A = F'(x*), B = F'(x), E = F'(x) — F'(x*). (6) 
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The last definitions, together with the latter inequality, imply that 

ll^tll^l^nptl^i, 

which, using that F'{x*) is injective, implies in view of Lemma [2] that F'(x) is injective. So, 
F'(x)*F'{x) is invertible and by definition of r we obtain that F'(x)*F'(x) is invertible for all 
x G B(x*,r). 

We already know that F'(x*) and F'(x) are injective. Hence, to conclude the lemma use 
definitions in ([6]) and then combine the above inequality and Lemma El □ 

Now, it is convenient to study the linearization error of F at point in Q, for which we define 
E F (x,y):=F(y)-[F(x)+F'(x)(y-x)}, y, x G fi. (7) 
We will bound this error by the error in the linearization on the majorant function / 

e f (t,u):=f(u)-[f(t) + f(t)(u-t)], t,u€[0,R). (8) 
Lemma 13. If \\ x — x* \\ < k, then there holds \\Ep(x, x*)|| < ej(||x — 0). 

Proof. Since B{x 1f , k) is convex, we obtain that x* + t(x — x*) G B(x*, k), for < t < 1. Thus, as 
F is continuously differentiable in f2, definition of Ep and some simple manipulations yield 

||-E>(x, x*)|| < / \\[F'(x) — F'(x* + t(x — x*))]\\ ||x* — x\\ dr. 
Jo 

^From the last inequality and the assumption ([2]), we obtain 

\\E F {x,x*)\\< [ [/'(||s-x»||) - f (t\\x - x4)] \\x-x4 dr. 
Jo 

Evaluating the above integral and using definition of ej, the statement follows. □ 
Define the Gauss-Newton step to the functions F by the following equality: 

S F (x) := -F'(x) t F(x). (9) 
Lemma 14. If \\x — x*|| < min{z^, k}, then 

no , ,ii «- Pe,(\\x - x.\\,0) + <f2cplf'(l\x-x,\\) + l] 
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Proof. Using Q, = and some algebraic manipulation, it follows from ([7|) that 

||S>(s)|| = ||F'(x)t (F(x.) - [F(x)+F'(x)(x* - x)}) - (F'(x)t - F'(xJ)F(x*) + (x* - x)|| 
< \\F'(x)i\\ \\E F (x,x*)\\ + ||F'(x)t - F'(x*)+||||F(x*)|| + ||x - x*||. 

So, the last inequality together with the Lemma \V2\ Lemma [131 and definition of c, imply that 

H s , x) « < Pe f (\\x - x.\\,0) ^c/3 2 [r(||x-x,||) + l] „ „ 

which is equivalent to the desired inequality. □ 

Lemma 15. Let Q C X be an open set and F : Q — > Y a continuously differentiable function. 
Let x* 6 £1, R > and c, f3, k as a definition in Theorem^ Suppose that F'(x*)*F(x*) = 0, 
F'(x*) is injective and there exists a f : [0, R) — > M continuously differentiable satisfying ([2]) ; hi, 
h2 and h3. Let a, oj\, 0J2, v> P o-nd r as in Theorem^ Assume that x £ -B(x*, r)\{x*}, i.e., 
< ||x — x*|| < r. Define 

x + = x + S, B(x)S = -F'{x)*F{x) +r, (10) 

where B{x) is an invertible approximation of F'{x)*F'(x) satisfying 

\\B{x)~ l F'{x)*F'{x)\\ < wi, ||S(x)- 1 F / (x)*F'(x) - J|| < w 2 , (11) 

and i/ie forcing term 9 and the residual r satisfy 

9cond{PF'{x)*F'(x)) < •&, \\Pr\\ < 8\\PF'(x)* F(x)\\, (12) 

with P an invertible matrix (preconditioner for the linear system in fjlOf) ). Then x+ is well defined 
and there holds 

\\ XM - X II < g I h a ):Ji3 [f'(\\ X - X *\\)\\x-x4-f(\\x-x4)} 2 

||x + s,||< (1 + 0)0*0 llx _ x42[1 _ p(mx _ x4) + 1)] \\x x„|| 

^ (l + #)cW2c/3 2 [/'(||x-x*||) + l] V 
+ ( v l|x-x,||[l- W (||x-x,|| ) + l)] + ^ + " 2 j »*-**»' fe = '^--- ( 13 ) 

/n particular, 

1 1 i2/j^( 1 1 1 1 1£< 3j 1 1 . 

Proof. First note that, as ||x — x*|| < r, it follows from Lemma [T2l that F'{x)*F\x) is invertible. 
Now, let i?(x) an invertible approximation of it satisfying (llip . Thus, x + is well defined. Now, as 
F 1 (x*)* F{x*) = 0, some simple algebraic manipulation and (fTUj) yield 

x+ - x* = x - x* - B{x)- 1 F'{x)* (F(x) - F(x*)) + £(x)-V 

+ S(x)- 1 F'(x)*F'(x) fF'(x,) t F(x,) - F'(x) t F(x, 
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Again, some algebraic manipulation in the above equation gives 

x+ - x* = B(x)- 1 F'(xyF'(x)F'(x^(F(x^) - [F(x) + F'(x)(x* - x)}) + B{ X y 1 r 
+ -B(x) -1 (F'(x)*F'(x) — -B(x)) (x — x* ) + B{x)- 1 F'{xyF'{x)[F'{x^F{x^) - F'(x) t F(x,)]. 

The last equation, together with ([7]) and (fTTj) . imply that 

||x+-x*|| < wi||F'(x) t ||||E F (x,x,)|| + ||5(x)-V||+w 2 ||x-x,||+a;i||F , (x) t -F , (x*) t ||||F(x,)||. 



On the other hand, using (J9|), (fTTj) and (|T2j) we have, by simple calculus, 

|| J B(x)- 1 r|| < || J B(x)- 1 p- 1 ||||Pr|| 

< 0|| J B(x)- 1 F , (x)*F , (x)||||(PF , (x)*F , (x))- 1 ||||PF / (x)*F , (x)||||F , (x) t F(x)|| 
<ux-d\\S F {x)\\. 

Hence, it follows from the two last equations that 

\\x + -x4 < wi||F , (x) t ||||F F (x,x,)|| +uj 1 ti\\S F (x)\\ +U3 2 \\x-x4 + Wl ||F'(x)t - F'(x*)t||||F(x*)||. 
Combining the last equation with the Lemmas [T2T [T3l and [TT1 we obtain that 
II iiwi-Ltt* ~ x*l|,0) + ^/3(/'(||x - x,||) + 1) 

x+-x* < l + tf/3^ — — — +ui# x-x* +w 2 x-x* . 

(If-»*II) + i) 

Now, using ([8]) and some algebraic manipulation, we conclude from the last inequality that 

f'(\\x - x4)\\x - x4 - f(\\x - x4) + V2cf3(f(\\x - x4) + 1) 



|x + — x*|| < (1 + d)f}u\ : 



l-/3(/'(||x-x,||) + l) 

+ — X*|| + Ul2\\x — X* 



which is equivalent to (|13p . To end the proof, note that the right hand side of (|13p is equivalent to 



ri , 9 x J'iW* ~ **\\)\\x - x4 - /(Ik - x4) + V2c(3{f{\\x - x4) + 1) 

(1 + ^ ||x-x,||[l- W (||x-x,||) + l)] + ^ + " 2 



On the other hand, as x G B(x*, ?")/{x*}, i.e., 0< ||x — x*|| <r< p we apply the Proposition [TT1 
with t = \\x — x4 t° conclude that the quantity in the bracket above is less than one. So, the last 
inequality of the lemma follows. □ 
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2.3 Proof of Theorem 

Now, we will produce the proof of Theorem [7J 

Proof. Since xq S B(x*, r)/{x*}, i.e., < ||xo — x*|| < r, by combination of Lemma [T2l last 
inequality in Lemma [T5l and induction argument, it is easy to see that {x^} is well defined and 
remains in B(x*,r). 

We are going to prove that {xk} converges towards x*. As, {xk} is well defined and contained 
in B(x*,r), applying Lemma [T5l with x + = x^+i, x = x^, r = r&, B(x) = B(xf,), P = Pk, and 
6 = 9k we obtain 

11**+! **|| < (1 + ^ ||x fc -x,P[l-/3(/'(||x fc -x,||) + l)] " 

+ [\\x k -x4[l- P(f(\\x k -x4) + l)] + «** + *»J 1^-^11' ,LI 
Now, using the last inequality of Lemma [T5l it is easy to conclude that 

||xfc — x*|| < ||x — x*||, fe = l,2.... (14) 
Hence, combining the last two inequalities with the last part of Proposition [10] we obtain that 

- x,ii < (i + ^ ^''^"li' 11 ^;,! 1 " 7 ' 11 ?,:^ 1 ' 1 !^ - **i' 2 

||x - x*|| 2 [i - /3(/'(||x - x*||) + l)] 

/ (l+^^c/j^rdixo-x.iD + i] v 

+ li Hh «7777ii i h , -,M +^l^ + ^2 x fc -x* , A: = 0,1,..., 

^ ||x - x*||[l - p{f{\\x - x*||) + 1)] y 

which is the inequality ([5]). Now, using (|14p and the last inequality we have 
||x^+i x* || ^ 

n i . ^ /^o ~ **ll)ll*o ~ x *\\- /(N ~ **ll) + yggMN ~ X *ID + i . .. ,q i .. , 1 ii r ,11 

(1+#MP 1] ttt; o, t ,i\\ \\\ i im hWi^+W 2 X fc -X* , 

Fo - **l|[i - PU (IN - x*||) + 1)] 

for all = 0, 1, . . .. Applying Proposition [TT1 with t = \\xq — x*|| it is straightforward to conclude 
from the latter inequality that {||x& — x*||} converges to zero. So, {x^} converges to x*. □ 



3 Special cases 

In this section, we present two special cases of Theorem [7J They include the classical convergence 
theorem on Gauss-Newton's method under the Lipschitz condition and Smale's theorem on Gauss- 
Newton for analytical functions. 
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3.1 Convergence result for Lipschitz condition 

In this section we show a correspondent theorem for Theorem [7J under the Lipschitz condition, 
instead of the general assumption ([2]). 

Theorem 16. Let Q C X be an open set, F : f2 — > Y a continuously differentiable function. Let 
£ Q, R > and 

c:= \\F(x*)\\, /?:= F'(x*) f , K := sup {t G [0, i?) : B(x*,t) C fi} . 

Suppose that F'(x*)* F(x*) = 0, F'(x*) is infective and there exists a K > suc/i t/iai 

a := V2cf3 2 K < 1, ||F'(x) - F'(y)|| < if||x - y\\, V x, y G B(x*, n). 

Take < i? < 1, < W2 < wi such that wi(a + ai? + i?) + <^2 < 1. £e£ 

f 2(1 -witf-ofc) - 2 v / 2c^/3 : Wl + 1 

r := mm < k, — ; : > . 

\ ' PK (2 + wi - - 2w 2 ) J 

Then, i/te inexact Gauss-Newton like methods for solving ([I]), with initial point xq G B(x*, r)\{x*} 

x k+1 = x k + S k , B{x k )S k = -F'(x k )*F(x k ) + r k , A; = 0,1,..., (15) 

with the following conditions for the residual r k , and the forcing term 9 k 

\\P k r k \\ < e k \\P k F'(x k )*F(x k )\\, < 6 k cond(P k F'{x k yF'(x k )) < 0, k = 0, 1, ... , 

where {P k } is an invertible matrix sequence (preconditoners for the linear system in (|15|) ) and 
B(x k ) is an invertible approximation of F'(x k )*F'(x k ) satisfying 

\\B(x k )- 1 F , {x k )*F'(x k )\\ < ui, \\B{x k )- 1 F'(x k )*F'(x k ) - I\\ < u 2 , k = 0, 1, . . . , 

is well defined, contained in B(x*,r), converges to x* and there holds 

x k+1 -x*\\ < — — ^ Xk-x* + — -+ui$ + u 2 \\x k - x* , 

2(1 - pK\\x Q - x«\\) y 1 - pK \\x - x*\\ J 

for all k = 0,1, ... . 

Proof. It is immediate to prove that F, x* and / : [0, /c) — >• R as defined by f{t) = Kt 2 /2 — t, 
satisfy the inequality ([2]), conditions hi and h2. Since yj2c[3 2 K < 1 the condition h3 also holds. 
In this case, it is easy to see that constants v and p as defined in Theorem [71 satisfy 

2(1 - urf - u> 2 ) - 2y / 2cif/?V(l + g) 
0< " = /MT (2+ a* -2a*) ^ =1 //^ 

as a consequence, < r = min{«;, p}. Therefore, as F, r, / and x* satisfy all of the hypotheses of 
Theorem[7J taking xq G B(x*, r)\{x*} the statements of the theorem follow from Theorem[7J □ 
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For the case $ = 0, the Theorem 1161 becomes: 

Corollary 17. Let il, C X be an open set, F : — >■ Y a continuously differentiable function. Let 
x* £ £1, R > and 



c := \\F(x* 



P:-- 



F'(x*) 



k := sup{t G [0,i?) : B(x*,t) C 0} 



Suppose that F' (x*)* F{x*) = 0, F'(x*) is injective and there exists a K > suc/i 

a := V2cf3 2 K < 1, - < K\\x — y\\, V x,y E B(x*, n). 

Take < ui 2 < suc/z i/iat u>ia + W2 < 1- £et 

r 2(1 - oj 2 ) - 2V2cK/3 2 uj 1 1 



r := mm < k, 



Then, the Gauss-Newton's like method for solving ([1]), with initial point xq E -B(x*, r)\{x*} 

x k+1 = x k + S k , B(x k )S k = -F'(x k )*F(x k ), A; = 0,1,..., (16) 
where B(x k ) is an invertible approximation of F'(x k )*F'(x k ) satisfying 

\\B(x k )- 1 F'(x k )*F'(x k )\\ < wi, \\B(x k )- 1 F'(x k )*F'(x k ) - L\\ < cj 2 , k = 0, 1, ... , 
is well defined, contained in B(x*,r), converges to x* and there holds 



I %k-\-l || ^ 



+ ^2 Xfe - X* 



2(1 - ||x - x*||) 11 *" - /3i^||x - x, 

/or aZZ = 0, 1, . . . . 

Note that letting c = in the above corollary, we obtain the Corollary 6.1 of [8]. 
3.2 Convergence result under Smale's condition 

In this section we present a correspondent theorem to Theorem [7] under Smale's condition. For 
more details see Smale [26] and Dedieu and Shub [TT] . 

Theorem 18. Let fi C X be an open set, F : £1 — > Y a continuously differentiable function. Let 
G fi, -R > and 



c := \\F(x* 



F'(x*) f , k := sup {t S [0, R) : B(x*, t) C £1} . 
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Suppose that F'(x*)*F(x*) = 0, F'(x*) is injective and 

l/(n-l) 



7 := sup 

n>l 



i?(")( 



nl 



< +00, 



a := 2\/2c/3 2 7 < 1. 



(17) 



Take < # < 1, < W2 < u>i suc/i i/mi wi(a + ai? + 1?) + W2 < 1. £ei a := (1 — $cji — W2), 
6 := (1 + i?)wi/9, a := 6 + 2a(l + /3) - \/2 7 /36c and 



r := mm < k, 



a 2 - 4a(l + /3)(a- 2V2c/3b~f) 



2a 7 (l + /3) 



Then, i/ie inexact Gauss-Newton like methods for solving ([T]) ; with initial point xq S B{x*, r)\{x*} 

x k+ i=x k + S k , B(x k )S k = -F'(x k )*F(x k ) + r k , A; = 0,1,..., (18) 

with the following conditions for the residual r k , and the forcing term 9 k 

\\P k r k \\ < 9 k \\P k F'(x k )*F(x k )l < 9 k cond{P k F' {x k )* F' {x k )) < 0, k = 0, 1, ... , 

where {P k } is an invertible matrix sequence (preconditoners for the linear system in (j 18[) ) and 
B(x k ) is an invertible approximation of F'(x k )*F'(x k ) satisfying 

\\B{x k )- 1 F'{x k )*F'(x k )\\ < wi, \\B{x k )- l F'(x k )*F'(x k ) - I\\ < u 2 , k = 0, 1, ... , 

is well defined, contained in B(x*,r), converges to x* and there holds 

(1 + fyutf-y 



+ 



(1 - j\\x - x*||) 2 - /?7(2||xo - x*|| - 7||x - x*|| 2 ) 
/ (l + i?)a;i\/2c/3 2 7(2 -7||xo-x,||) 



I %k I 



Ul'd + U>2 \\x k - X* 



(1 - 7||x - x*||) 2 - /3 7 (2||x - x*|| - 7 ||x - x*|| 2 ) 

for all k = 0,1, ... . 

We need the following result to prove the above theorem. 

Lemma 19. Let ft CX be an open set, F : — > Y an analytic function. Suppose that x* G and 
B(x*, I/7) C f2, where 7 is defined in (|17p . Then, for all x € B(x*, l/ 7 ) i/iere ZioZds 

||F"(x)|| ^ 2 7 /(l - 7 ||x-x,||) 3 . 

Proof. See the proof of the Lemma 21 of [16]. □ 
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The next result gives a condition that is easier to check than condition ([2]), whenever the 
functions under consideration are twice continuously differentiable. 

Lemma 20. Let Q C X be an open set, x* G and F : £1 — > Y be twice continuously on Q. If 
there exists a f : [0, R) — > R twice continuously differentiable such that 



\\F"(x)\\^f"(\\x-x4), 
for all x G £1 such that \\x — x*\\ < R. Then F and f satisfy ([2]). 
Proof. See the proof of the Lemma 22 of [16J. 

[Proof of Theorem I18j . Consider the real function / : [0, 1/7) 

/(*) = T^— ~ 2t- 
1 — 71 

It is straightforward to show that / is analytic and that 



(19) 



□ 



defined by 



/(o) = 0, f\t) = i/{\- lt f 



/'(()) = -1, f (t) = (2 7 )/(l- 7 t) 3 , / n (0) = n! 7 



ra-1 



for n > 2. It follows from the last equalities that / satisfies hi and h2. Since 2^/2c/3 2 7 < 1 the 
condition h3 also holds. Now, as f"(t) = (2 r y)/(l — "ft) 3 combining Lemmas 1201 1191 we conclude 
that F and / satisfy ([2]) with R = I/7. In this case, it is easy to see that constants v and p as 
defined in Theorem [71 satisfy 



< p 



a 2 - 4a(l + /3)(a- 2V2~ C pbj) 



2a 7 (l + (3) 



< v = ((1 + p) - V/3(l + /3))/(7(l + ?)) < 1/7, 



and as a consequence, < r = min{K, p}. Therefore, as F, a, f and x* satisfy all hypotheses of 
Theorem[7J taking xq S B(x%, r)\{x*}, the statements of the theorem follow from Theorem[JJ □ 

For the case t? = 0, the Theorem 1181 becomes: 

Corollary 21. Xei Q QH be an open set, F : f2 — > Y a continuously differentiable function. Let 
x* £ £1, R > and 



c :- 



\F(x* 



/3:= F'(x*)t 



K := sup{t G [0,i?) : B{x*,t) C 0} 



Suppose that F' (x*)* F{x*) = 0, F'( injective and 

l/(n-l) 



7 := sup 

n>l 



< +OO, 



a := 2\/2c/3 2 7 < 1. 
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Take < w 2 < oj\ such that uj\a + uj-i < 1. Let a := wi/3 + 2(1 — u> 2 )(l + /3) — \/27/3 2 wic and 



r := mm < 



a - V a 2 - 4(1 - w 2 )(l + /3)(1 - w 2 - 2^2c^uj x1 ) 



2(l-^ 2 ) 7 (l + /3) 

Then, the Gauss-Newton's like method for solving (pQ), with initial point xq G B{x if , r)\{x*} 

x k +i = x k + Sk, B(x k )Sk = -F'(x k )*F(x k ), A; = 0,1,..., 
where B(x k ) is an invertible approximation of F'(x k )*F'(x k ) satisfying 

\\B(x k )- 1 F'(x k )*F , (x k )\\ < u h \\B(x k )- 1 F'(x k )*F'(x k ) - I\\ < u 2 , k = 0, 1, . . . , 
is well defined, contained in B(x*,r), converges to x* and there holds 

\\x k l - x II < — ^ \\x k - x || 2 

+ * ~(1 - j\\xq - x*||) 2 - P"f(2\\x - a:* 1 1 - 7lko - ^*|| 2 ) 

+ / ^i^ C /3 2 7(2- 7 ||xo-^||) + \ _ ^ 

y(l - 7||x - a;*||) 2 - /37( 2 ||a;o - ^*|| - 7lko - 2;*|| 2 ) J 

for all k = 0,1, ... . 

Note that letting c = in the above corollary, we obtain the Example 1 of [8] . 

4 Final remark 

The Theorem [7| gives an estimate of the convergence radius for inexact Gauss-Newton like methods. 
In particular, for •& = uj\ = and w 2 = 1 is shown in Ferreira et al. |16j . that r is the best possible 
convergence radius. 

Another detail is that, as pointed out by Morini in [23] if preconditioning P k , satisfying 



\\Pkr k \\ < 9k\\PkF'(x k )*F(x k )\\, (20) 

for some forcing sequence {6 k }, is applied to finding the inexact Gauss- Newton steep, then the 
inverse proportionality between each forcing term 9 k and cond(P k F'(x k )*F(x k )) stated in the fol- 
lowing assumption: 

< 9 k cond(P k F'(x k )*F(x k )) < 0, k = 0, 1, . . . , (21) 

is sufficient to guarantee convergence, and may be overly restrictive to bound the sequence {0 k }, 
always such that the matrices P k F'(x k )*F(x k ), for k = 0, 1, . . . , are badly conditioned. Moreover, 
9 k does not depend on cond(F' (x k )* F(x k )) but only on the cond(P k F' \x k )* F(x k )) and a suitable 
choice of scaling matrix P k leads to a relaxation of the forcing terms. 
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